Search for: All records

Abstract The rise of Large Language Models (LLMs) and generative visual analytics systems has transformed data‐driven insights, yet significant challenges persist in accurately interpreting users analytical and interaction intents. While language inputs offer flexibility, they often lack precision, making the expression of complex intents inefficient, error‐prone, and time‐intensive. To address these limitations, we investigate the design space of multimodal interactions for generative visual analytics through a literature review and pilot brainstorming sessions. Building on these insights, we introduce a highly extensible workflow that integrates multiple LLM agents for intent inference and visualization generation. We develop InterChat, a generative visual analytics system that combines direct manipulation of visual elements with natural language inputs. This integration enables precise intent communication and supports progressive, visually driven exploratory data analyses. By employing effective prompt engineering, and contextual interaction linking, alongside intuitive visualization and interaction designs, InterChat bridges the gap between user interactions and LLM‐driven visualizations, enhancing both interpretability and usability. Extensive evaluations, including two usage scenarios, a user study, and expert feedback, demonstrate the effectiveness of InterChat. Results show significant improvements in the accuracy and efficiency of handling complex visual analytics tasks, highlighting the potential of multimodal interactions to redefine user engagement and analytical depth in generative visual analytics. 

Distributed market structures for local, transactive energy trading can be modeled with ecological systems, such as mycorrhizal networks, which have evolved to facilitate interplant carbon exchange in forest ecosystems. However, the complexity of these ecological systems can make it challenging to understand the effect that adopting these models could have on distributed energy systems and the magnitude of associated performance parameters. We therefore simplified and implemented a previously developed blueprint for mycorrhizal energy market models to isolate the effect of the mycorrhizal intervention in allowing buildings to redistribute portions of energy assets on competing local, decentralized marketplaces. Results indicate that the applied mycorrhizal intervention only minimally affects market and building performance indicators—increasing market self-consumption, decreasing market self-sufficiency, and decreasing building weekly savings across all seasonal (winter, fall, summer) and typological (residential, mixed-use) cases when compared to a fixed, retail feed-in-tariff market structure. The work concludes with a discussion of opportunities for further expansion of the proposed mycorrhizal market framework through reinforcement learning as well as limitations and policy recommendations considering emerging aggregated distributed energy resource (DER) access to wholesale energy markets. 

Identifying people in photographs is an important task in many fields, including history, journalism, genealogy, and collecting, but accurate person identification remains challenging. Researchers especially struggle with the “last-mile problem” of historical person identification, where they must make a selection among a small number of highly similar candidates. We present SleuthTalk, a web-based collaboration tool integrated into the public website Civil War Photo Sleuth which addresses the last-mile problem in historical person identification by providing support for shortlisting potential candidates from face recognition results, private collaborative workspaces, and structured feedback. 

Identifying historical photographs of people can generate significant cultural and economic value, but misidentifications can cause harms such as falsifying the historical record, spreading disinformation, and feeding conspiracy theories. In this paper, we introduce DoubleCheck, a quality assessment framework based on the concepts of information provenance and stewardship for verifying historical photo identifications. We built and evaluated DoubleCheck on Civil War Photo Sleuth (CWPS), a popular online community dedicated to identifying photos from the American CivilWar era (1861- 65) using facial recognition and crowdsourcing. 

Online extremism can quickly spill over into the physical world and have dangerous consequences, as when rioters attacked the U.S. Capitol on January 6, 2021. While information and communication technologies have enabled extremists to plan and organize violent events, they have also enabled collective action by others to identify the perpetrators and hold them accountable. Through a mixed-methods case study of Sedition Hunters, a Twitter-based community whose goal is to identify individuals who took part in the Capitol attack, we explore: 1) how the community formed and changed over time; 2) the motives, ethos, and roles of its members; and 3) the methods and software tools they used to identify individuals and coordinate their activities. 

Identifying people in historical photographs is important for interpreting material culture, correcting the historical record, and creating economic value, but it is also a complex and challenging task. In this paper, we focus on identifying portraits of soldiers who participated in the American Civil War (1861-65). Millions of these portraits survive, but only 10-20% are identified. We created Photo Sleuth, a web-based platform that combines crowdsourced human expertise and automated face recognition to support Civil War portrait identification. Our mixed-methods evaluation of Photo Sleuth one month after its public launch showed that it helped users successfully identify unknown portraits. 

Facial recognition systems pose numerous ethical challenges, yet little guidance is available for designers. We explore these challenges in a three-step design process to create Civil War Twin, an educational web application where users can discover their lookalikes from the American Civil War era while learning more about both history and facial recognition.